The present invention relates to a motion vector search in predictive coding of a digital picture.
An inter-frame predictive coding may be given as a method of reducing data by making use of time-base directional redundancy of a motion picture among methods of compressing a data quantity of motion picture data. According to MPEG-1/2 defined as an international standardizing system for digital motion picture coding, both of a forward prediction and a bidirectional prediction can be availed.
Forward Prediction
As shown in FIG. 1, a motion between a frame in the past and a frame at present is detected as a forward motion vector, and the frame at present is generated referring to the frame in the past (FIG. 1). This motion vector searching is done by a macroblock consisting of, e.g., 16.times.16 pixels. A difference in terms of a spatial position from an area exhibiting a largest correlation with respect to a frame with a prediction effected in the forward reference frame, is detected as a motion vector.
Bidirectional Prediction
As shown in FIG. 2, according to a bidirectional prediction, the present frame is generated referring to both or one of the past frame and the prospective frame.
According to these predictive systems, the motion is sought in every area consisting of a plurality of pixels, and a quantity of data for coding the motion in every area as vector data instead of the pixel data can be remarkably reduced. If there is a difference between an original picture and a picture (a predictive picture) composed by use of the motion vectors, data about this difference is coded together with the vector data. Namely, a larger quantity of data can be reduced as the difference between the predictive picture and the original picture becomes smaller. If a motion vector search range is the same, the bidirectional prediction exhibits a higher possibility that a reference picture approximate to the picture serving as a prediction target than by the forward prediction, and hence generally a predictive efficiency becomes higher by using the bidirectional prediction than by the forward prediction. Hereinafter, a frame that can be generated without any data about front and rear pictures, is termed an I-picture (Intra Coded Picture). A frame coded by use of the only forward prediction from past picture is referrd to as a P-picture. (Predictive Coded Picture). A frame coded by use of the forward/backward bidirectional predictions from past/prospective pictures is called a B-picture. (Bidirectionally Predictive Coded Picture).
In general, the motion vector search is attained by sectioning the original picture into a plurality of areas on the coding basis, comparing these areas with a reference picture while being shifted pixelwise in parallel in the horizontal and vertical directions, and thereby seeking a position having the highest correlation. A range for shifting the picture in parallel to obtain the degree of correlation is limited by a performance of transferring the data between a processor for calculating the degree of correlation and the frame memory for storing the reference picture. FIG. 7 is a block diagram illustrating a construction of the coding apparatus using a conventional vector searching method, and FIG. 9 is a flowchart thereof. Further, FIG. 3 shows an example of a coding sequence and the conventional motion vector search.
Referring to FIG. 7, normally a frame memory 71 is so used as to be sectioned into an area 71a used for inputting pictures, an area 71b for storing a coding target picture, an area 71c for storing a forward reference picture, and an area 71d for storing a backward reference picture. A degree-of-correlation calculating unit 72 performs a motion vector search. This degree-of-correlation calculating unit 72, upon inputting coded area data of a coding target picture, designates areas for the forward reference picture and the backward reference picture, and there are inputted pixel data of the reference areas from these reference pictures.
Herein, when coding the P-picture, the data inputted to the degree-of-correlation calculating unit 72 are only pixel data (71b) of the area becoming a coding target of the original picture, and pixel data (71c) of the reference range of the forward reference picture. Further, when coding the B-picture, the reference picture turns out to be the pixel data (71c, 71d) in both of the forward and backward reference ranges.
Then, the degree-of-correlation calculating unit 72 outputs, to a coding unit 73, the forward motion vector in the case of coding the P-picture, and the forward and backward motion vectors in the case of coding the B-picture. The coding unit 73 generates the reference picture with reference to a local decode picture frame memory 74 on the basis of these pieces of vector data, and executes inter-frame predictive coding. The thus coded picture is simultaneously decoded and recorded in the local decode picture frame memory 74 so as to be used as a reference picture thereafter.
FIG. 9 shows a processing flow in the coding apparatus shown in FIG. 7. At first, when determining a picture type, e.g., I-picture coding, P-picture coding or B-picture coding (901), there must be a standby status till the required picture data are inputted to the frame memory 71 (902). Upon inputting the picture data, the picture data of the coding target area is extracted out (903), and the processing is diverged depending on the kinds (the I-picture, or the P-picture, or the B-picture) of the relevant picture data (904, 905). More specifically, in the case of the I-picture, the processing moves directly to a coding step (909). In the case of the P-picture, the motion vector search is executed with the I-or P-picture just anterior serving as a reference frame (908), and thereafter the coding step is implemented, (909). Furthermore, in the case of the B-picture, the motion vector search is effected with the I- or P-picture just anterior serving as a forward reference frame (906). Moreover, the motion vector search is carried out with the I- or P-picture just poterior serving as a backward refernce frame (907), and thereafter the coding process is executed.
A series of processes discussed so far are repeated per picture (910).
FIG. 3 is a sequence diagram on the assumption that a single frame to which the bidirectional prediction is applied is provided behind the frame to which the forward prediction is applied.
Referring to FIG. 3, the coding process is executed by use of a frame memory composed of five areas, an interval for inputting each frame is fixed according to the general video signal specifications, and an interval for coding each frame is also fixed. Therefore, for example, a time I and a time II in the coding process are equal to each other.
Because of the necessity for coding a prospective frame in advance in order to effect the bidirectional prediction, an input sequence of the picture data is different from a coding sequence. The five areas of the frame memory are needed for rearranging this processing sequence.
As shown in FIG. 3, according to the prior art, the motion vector search is implemented just before coding irrespective of the forward/bidirectional predictions. It was a general practice that a P1 motion vector search (P1f) and the coding process are executed for the time I, while B2 bidirectional motion vector searches (B2f, B2b) are performed for the time II.
By the way, a plurality of frame memories are constructed typically of inexpensive DRAMs in order to process a large quantity of data when dealing with the picture data. In this case, it follows that the plurality of frame memories share a single line of data bus. Further, a reduction in costs for manufacturing the motion vector search modules entails decreasing the number of data I/O terminals for the frame memories. If the plurality of frame memories share the I/O terminals, however, a degree of freedom of accessing to the frame memory is restricted because of a conflict of accessing, and hence, when the number of terminals is excessively decreased, the quantity of transferable data can not be secured.
Moreover, the data quantity transferrable per unit time is restricted due to a limit in terms of data write/read speeds of the memory device.
On the other hand, it is required for surely searching the motion vector that the vector search be executed in a relatively wide area of the reference picture. Especially in the B-picture motion vector search, the forward and backward reference pictures are needed, and therefore the reference picture data the quantity of which is twice that of the picture with only the forward prediction effected, are required of the processor for obtaining the degree of correlation between the pictures. A large amount of accesses to the frame memories stored with the reference picture data are necessary for reading the above data.
As obvious from FIG. 3, according to the prior art, it is required that both of the motion vector search and the coding process be executed within a fixed time to perform the motion vector search just before coding. In this case, as a time spent for the motion vector search is limited, if a vector search range is taken large on the reference picture, it is impossible to effect the accesses to the frame memories enough to transfer the necessary data. This results in a difficulty to secure the sufficient motion vector search range.
Further, if the B-picture is frequently used, there must be inevitably a longer distance between the P-picture and the refernce frame which the P-picture references. Accordingly, there arises a necessity for taking a broader motion vector search range. In this case also, however, the accessing to the frame memories similarly increases in quantity, and this conduces to a problem of being incapable of securing the sufficient motion vector search range.